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Traffic modeling of communication networks such as Internet has become a very important field 
of research. A number of interesting phenomena are found in measurements and traffic simulations. 
One of them is the propagation of congestion waves opposite to the main packet flow direction. The 
purpose of this paper is to model and analyze packet congestion on a given route and to give a 
possible explanation to this phenomenon. 



In the recent past many aspects of computer data 
traffic on the Internet have been investigated. Signa- 
tures of long-range correlations Q, scaling[l|, chaos[3 
and phase transitions have been found. In Ref.[5j 
1//" noise has been observed in the time series of round 
trip times similar to those observed in highway traffic 
measurements 6J. Since the slowing down and accelera- 
tion of packet flow in computer networks are very similar 
to those of cellular automaton models of cars in highway 
traffic m, it has been argued that a valid analogy ex- 
ists between these subjects which later has been demon- 
strated quantitatively j9l One of the most peculiar 
features of car traffic and flow of granular media is the 
propagation of density waves 'llj. In car traffic stop-and- 
go type congestion waves propagate against the direction 
of the flow. In Ref.^^l the authors made an attempt 
to observe the propagation of congestion in the routers 
of a real computer network. They studied the spatio- 
temporal correlations of the level of congestion in routers 
and showed that congestion can propagate from a heav- 
ily loaded router to one of its empty neighboring routers. 
Today computer networks can be studied with the help 
of network simulators, which has been developed by en- 
gineers. These tools enable us to assemble any computer 
network configuration, to use the most commonly used 
packet sending mechanism TCP/IP protocol and to em- 
ulate its real behavior without building the system from 
hardware components. These simulators can imitate the 
behavior of hardware elements (computers, routers, lines 
etc.) accurately so that the results of the simulations are 
nearly identical with those obtained from measurements. 
One of the most popular tools is the Berkeley Network 
Simulator (ns-2.26 14J) which is used in this paper. 

In this letter we study the network traffic generated 
in a unidirectional ring of identical routers connected as 
shown in the left part of Fig. ^ The ring geometry mim- 
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FIG. 1: The ring structure. Left, Network simulator: The 
TCP agent at site i continuously transfers data trough the 
ring of network routers connected with constant capacity C 
lines with time delay r. The packets not processed yet by 
the routers are queued in buffers of size B. The stream of 
data can traverse in clockwise direction only to arrive at the 
destination node at site i — 1. In the simulations a realistic 
set of parameters has been set (C — 10^ bits/s, r — .031 s, 
B = 300 packets, P = 4416 bits, = 10). Right, Continuous 
model: Each TCP is modeled with its sending rate Xi and its 
traffic flow traverses the ring topology. On the link connecting 
sites i — 1 and i all the sending rates of traversing TCP flows 
are summed up to yield the total link utilization. 



ics periodic boundary conditions. This way we can study 
the propagation of congestion in an isolated, clean setup, 
which makes it possible to compare the results to those 
obtained in cellular automaton, car and granular traffic 
simulations. We show that this system drives itself into a 
critical congested state in a self-organized way. Both the 
position of the congested router and the packet sending 
activity at the sites propagate against the direction of the 
packet flow. The profile of the congestion wave can be 
reconstructed from the activities of the computers con- 
nected to the ring. The velocity of the congestion wave 
can be measured. Then a simple model of the system is 
introduced which is able to explain the main features of 
the congestion wave such as shape and speed. A formula 
for the velocity of the congestion wave is obtained and 
tested. 

In our model system the ring is formed by N identi- 
cal routers which can forward packets in clockwise direc- 
tion. Routers are connected with a line of capacity C 
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(measured in data bits per second) with a constant for- 
warding delay r (measured in seconds). The incoming 
data flow of a router can temporarily exceed the capac- 
ity of the outgoing line. To avoid data loss in this sit- 
uation the router contains a buffer of size B (measured 
in data packets) where packets can be stored. The com- 
puters are instructed to send data persistently to their 
anti-clockwise neighbors, so that the packets traverse the 
longest possible route in the ring. The traffic is "granu- 
lar" as computers send in data packets of size P (mea- 
sured in bits). The data traffic of computers is controlled 
by the TCP /IP protocol 7] . This protocol ensures that 
the data packet-sending rate is decreased whenever con- 
gestion occurs and that it is increased when there is an 
available unused capacity in the system. The control al- 
gorithm became quite sophisticated over the years. Here 
we can only sketch the main properties of the most pop- 
ular version used currently. 

After establishing connection between two computers 
over the network TCP algorithm regulates the packet- 
sending rate. First a single packet is sent out. Upon 
receiving that packet the receiver acknowledges the ar- 
rival of the packet by sending back a small size acknowl- 
edgement packet (ACK). The time elapsed between the 
sending out of a packet and receiving the corresponding 
ACK is called round trip time (RTT). The TCP main- 
tains an internal variable, the Congestion Window (w), 
which is used to control the number of packets sent out 
when the ACK is received. It starts with the initial value 
w = 1 and then it is increased according to w >—>■ w + 1 / w 
each time an ACK is received. Two new packets are sent 
out if the congestion window crosses an integer value and 
only a single packet otherwise. This way the integer part 
of the window [w] gives the number of sent but not yet ac- 
knowledged packets in the network. Assuming constant 
RTT during this process, the congestion window and the 
number of packets out in the network are increased lin- 
early in time. This process lasts until a packet is lost 
somewhere in the network, indicating congestion. As a 
response the packet-sending rate should be decreased. 
So, the TCP reduces the value of the congestion win- 
dow w >—>■ [3w (/3 < 1) and does not send out any new 
packets in response to ACKs until the number of still 
unacknowledged packets decreases to the integer part of 
the new (reduced) value of the congestion window. After 
that the packet-sending algorithm returns to the original 
linear increase phase described above. 

In case the congestion window variable is large we can 
neglect its granularity and can treat it as a continuous 
variable. In the linear increase phase the window variable 
is increased by one in each RTT period and its dynamics 
can be well approximated with the differential equation 
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where TuTrit) is the actual value of the RTT measured 
by the TCP. In case of packet loss the window is reduced 



w{t+) = Pw{t^), 



(2) 



where t±—t±Q denotes the time right after and before 
the packet loss. The sending rate X (bit/sec) in the con- 
tinuous approximation can be estimated as the amount 
of data sent within an RTT, X = Pw /Trtt- Neglecting 
the change of RTT on the scale of RTTs the sending rate 
satisfies the following pair of equations: 



dX 
~dt 



P 



X{t^) = px{t-) at packet loss. 
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where typically /3 = 1/2. 

Next, we present the results of our simulation study 
carried out with the network simulator. The geometry 
and parameters of the setup is on the left part of Fig. ^ 
which is essentially the same as we could have observed 
in real hardware realization. Fig. |21 shows the spatiotem- 
poral diagram of the congestion wave occurring in the 
network simulator. One can see that after a short initial 
transient (up to 500 s) the pattern remains stable and 
propagates in anti-clockwise direction. In this respect it 
resembles the congestion propagation in car traffic. The 
speed of the congestion wave pattern is almost constant. 
Its average can be determined by measuring the average 
speed of the center of mass of the pattern. This should 
be carefully defined in the present situation as the sys- 
tem is spatially periodic. Taken into account the periodic 
boundary conditions the center of mass of the pattern can 
be defined by (i)(i) = aig{i:fSo^Xj{t)e'^^''^^'>^) . The 
speed of the pattern is the time derivative of this quan- 
tity. 

Once the speed of the pattern is determined we can 
analyze the shape of the profile. Representing the send- 
ing rates (t) in co- moving coordinates z' relative 
to the center of mass we recover the shape of the travel- 
ing wave pattern. Averaging the new series in time the 
profile of the front emerges as in Fig. |31 

While the continuous equations constitute gross sim- 
plification of the original TCP dynamics, the main prop- 
erties of the traveling wave can be recovered from them 
with some additional assumption made on the packet loss 
process as we show next. 

The bandwidth Ci_i(i) utilized on the link connecting 
nodes i — 1 and i is the sum of sending rates of TCPs 
whose traffic flows through that link. In our case the 
flows of all TCPs traverse that link except the one start- 
ing at node i and ending at node i — 1: 
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X,{t)=Y,X,{t)-X,{t), (5) 



where site i — N is identifled with site i — Q due to 
periodicity. The traffic of ACK packets emanating in 
i — \ and absorbed in i is low due to the small size of ACK 
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FIG. 2; Spatiotemporal diagram of congestion propagation. 
The horizontal axis is the real time (in seconds) of the system 
and the site index (i) is on the vertical axis. In this simulation 
the number of sites was N = 10. Notice, that the sites i — 
and i = N — 1 are neighbors in the ring topology. The sending 
rate Xi is color-coded according to the scale indicated at the 
right side of the figure. Deep blue indicates very low sending 
rate due to high congestion. It can be seen that the most 
congested site propagates in anti-clockwise direction with al- 
most constant speed, while the packet traffic itself is clockwise 
directed. 



in batches. As a response, TCPs send out data packets 
also in batches. Then obviously the TCP that ejects this 
burst of data packets directly into an almost saturated 
buffer will lose packets in the process. 

The mechanism described above is responsible for the 
emergence of the congestion wave in the system. The 
TCP at site i* — 1 suffers packet losses repeatedly until its 
sending rate _i becomes smaller than . From then 
on Xi*_i will be the lowest in the system, link utilization 
Ci*_2 will be the highest and TCP at site i* — 2 suffers 
the packet losses. This way congestion propagates site 
by site anticlockwise in the system. After several rounds 
of congestion propagation the propagating front of Fig[3| 
emerges. 

The shape of the front is linear with a sharp drop con- 
necting its ends. We can determine the parameters of 
the linear front in our model. Let the nodes forming the 
linear part of the front range from to iV — 1 and let the 
minimum sending rate first be at the 0th node 
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as it is shown in Fig. ^ that illustrates the evolution of 
a system containing N = 5 nodes. The packets start 
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FIG. 3: The shape of the traveling wave profile. It has been 
determined by averaging the time series in co-moving spatial 
coordinates. 



packets and its contribution to traffic can be neglected 
(see also the right part of Fig. Congestion and packet 
loss occur in the system whenever one of the bandwidths 
Ci{t) reaches the link capacity C. According to (jsj the 
largest link utilization Ci{t) is at site i = i* — 1 where 
i* is the site where the sending rate Xi* (t) is the lowest. 
We then have to investigate which TCP flow will lose 
packet on link i* — 1. In principle all the TCP flows 
traversing the congested link can lose packets, so only 
the TCP at site i* is immune. Our observation is that 
the TCP flow starting at the actual congested link (with 
sending rate experiences the packet loss almost 

surely. This is due to the fact that the processing of 
the small acknowledgement packets is faster than that of 
data packets and they tend to queue up and to arrive 
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FIG. 4: Sending rate evolution in time. N = 5, I 
decreases until it reaches the value of Xo- 
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to be dropped at site — 1. We start our description 
at the first packet drop, which occurs when Cn-^i{0) = 
J2^=o^X,{0) - Ao(0) = C holds. This initial condition 
gives the first condition for the initial shape of the front 



C^{N-l)[a+^b 
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Immediately after the packet drop the sending rate at site 
— 1 decreases to AAr_i(-|-0) — /?A7v-i(0), while the 
rest of the sending rates stay unchanged. Then the send- 
ing rates increase in a uniform manner with an amount 
X\ = X^ + {1-I3)Xn-i/{N-1) until the next packet loss 
occurs. In particular, from before the first packet loss un- 
til the second packet loss the sending rate at site A^ — 1 
changes to X'^-i = (?XAr_i(0), where q = /3 + -^^j. 
This process is then repeated I times until the sending 
rate q'^Xn-i becomes lower than the actual value of Xq. 
This way the wave moves one site to the left, while its 
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linear shape is preserved as the sending rates of all nodes, 
except A'^ — 1, increase with b. From Eq.|31one can calcu- 
late the time needed for this Tp — bT^rprp/P. Accordingly 
the speed of the congestion wave is v :— l/Tp (measured 
in site/sec). 

The formula derived for the speed of the congestion 
waves can be tested against real data produced by the 
network simulator. In the simulation one can measure 
the average round trip time observed by the TCPs, the 
mean slope of the linear part of the front b and the speed 
of the front. On the main part of Fig. |31one can see that 
the measured values satisfy the relation vbT^rprp/P = 1 
very well. 
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FIG. 5; Testing the relation vbTlj.j./P = 1 for various sys- 
tem sizes A'' = 5 — 20 in the network simulator. After the end 
of transients the values of b, Trtt and v have been obtained 
by averaging over samples of 5000 sec. Inset: Testing Eq.|H]in 
the network simulator. Average values of b are plotted against 
the results of Q with q' obtained from the simulation. 



With the help of Q and ® the slope of the front in 
the model can also be directly expressed as 



b = 



2C(l-gO 



{N ~1){N -2){q^ + 1)' 



(8) 



where I should be determined independently. Our math- 
ematical model allows several positive integer values of I 
with an upper bound due to the positiveness of b. We 
found that only the largest possible I value is stable 
against small perturbations of the wave front. Systems 
started at lower I values always shift towards a greater 
value of /. In the network simulator we always observed 
the realization of the most stable (the highest possible I) 
solutions of the model. In the inset of Fig. [3 we compare 
the measured values of b with (jS)). We again find good 
agreement. 

As a summary we showed that congestion waves are 
formed naturally in the data traffic of today's computer 
networks. The mechanism behind the wave formation is 
that packet losses occur most likely in computers nearest 
to the site of the actual congestion and other computers 
sharing the congested link increase their sending rates, 
moving the site of the congestion one site downstream. 
This basic mechanism is quite general and can create 
congestion moving against the direction of the data traf- 
fic in more complicated geometries. A formula for the 
speed of the congestion wave has been derived in a simple 
ring topology and network simulations have confirmed it. 
Such formulas can be developed for more complicated ge- 
ometries, which is our next research goal. 

The authors thank the National Science Fund, Hun- 
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